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Introduction 



The National Accessible Reading Assessment Projects (NARAP) is a collaborative effort 
that is conducting research to make large-scale assessments of reading proficiency more 
accessible for students who have disabilities that affect reading. There are two projects 
that comprise the NARAP partnership, Designing Accessible Reading Assessments 
(DARA) and the Partnership for Accessible Reading Assessment (PARA). DARA and 
PARA are working together and independently on research studies in support of the 
NARAP goals. 

NARAP s Goal 1, which the two projects worked on together, was to craft a definition 
of reading to support the development of a valid reading assessment for all students, 
including those with disabilities that affect reading. To this end NARAP formed a 
Definition Panel that included 15 national experts who had served on previous reviews 
or reading definition panels or who were experts in special education and various areas 
of disabilities research and education. The charge given to the panel of experts was 
to draw on the existing research base and five national reports to craft a definition of 
reading and reading proficiency that would press previous definitions and serve as the 
basis for the development of high quality reading assessments that are accessible for all 
students. 

The panel had a single face-to-face meeting in January 2005 that resulted in a high- 
level definition of reading (Definition A). A revision of this definition (Definition B) 
was constructed after the face-to-face meeting by a subcommittee initially charged 
with fleshing out the initial definition to include reading proficiency or proficiency 
statements. 

Definition B was presented to the full Definition Panel as an alternative to Definition 
A. During the e-mail discussion of the definitions, the majority favored the original 
Definition A. However the Panel did not reach consensus and thus did not move 
forward with the next steps of creating a definition of reading proficiency or 
proficiency statements for 4th and 8th grade students. Concerns raised about the two 
definitions stemmed from multiple perspectives about important components that 
individuals thought should be in the definition, including: (a) the role of decoding and 
comprehension or developing understanding about what is read, (b) what constructs 
define the act of reading vs. instructional aspects of reading such as end goals and 
purposes for reading as a component along with decoding and comprehension or 
understanding, and (c) the impact of various disabilities on how reading is defined. 

NARAP s Executive Committee studied the work of the Definition Panel and sought 
to find a compromise between the two definitions. They offered Definition C as a third 
alternative. The three definition statements that were drafted are: 
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Definition A 



Reading is decoding and understanding written text. Decoding requires translating the 
symbols of writing systems (including braille) into the spoken words they represent. 
Understanding is determined by the purposes for reading, the context, the nature of the 
text, and the readers’ strategies and knowledge. 

Definition B 

Reading is decoding and understanding text for particular reader purposes. Readers 
decode written text by translating text to speech, and translating directly to meaning. 

To understand written text, readers engage in constructive processes to make text 
meaningful, which is the end goal or product. 

Definition C 

Reading is the process of deriving meaning from text. For the majority of readers, this 
process involves decoding written text. Some individuals require adaptations such 
as braille or auditorization to support the decoding process. Understanding text is 
determined by the purposes for reading, the context, the nature of the text, and the 
readers’ strategies and knowledge. 

Obtaining input on the working definitions from many stakeholders with a variety of 
viewpoints on reading was an important step in support of Goal 1. Given the need to 
allow for many potential reactions to the definitions we decided that the open dialogue 
found in focus groups would best serve the feedback process. 

There were two main purposes to the focus groups. One purpose was to provide 
feedback to the Definition Panel from members of the reading, disability, and 
educational measurement communities. A second purpose was to provide evidence 
of support from those communities for the definition that would be adopted, as stated 
as a requirement in the initial grant proposal. In addition to these primary purposes, 
the NARAP project also hoped that the focus groups would help to develop interest in 
the NARAP project by engaging members of the reading, disability, and educational 
measurement communities. 



Method 

Two types of focus groups were held — traditional (face-to-face) and teleconference 
with Web support. DARA and PARA worked together to develop the basic protocols 
and collaborated on planning and recruitment. DARA took the lead on the traditional, 
face-to-face focus groups and PARA took the lead on the teleconference and Web-based 
sessions. The organizations on the NARAP General Advisory Gommittee that best 
represented the specific disability groups of interest to the project formed the pool of 
potential focus groups. 
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Face-to-face focus groups were held concurrently with large conferences in the spring 
of 2005 to take advantage of a hroad constituency of educators already congregated 
in a single location for a given time period. Participation was open to all conference 
attendees. However, some conferences were so large that only a subset were actually 
contacted. Potential participants were usually contacted via e-mail and an electronic 
mailing list, in collaboration with the conference or General Advisory Committee 
(GAG) members. Multiple sessions were scheduled during each conference. Conference 
attendees were sent a brief e-mail inviting those interested to pre-register for a session 
via a Web site hosted at ETS, with times and dates specific to each conference. Potential 
participants would indicate their availability and some background information. Groups 
were then formed based on participant characteristics and time availability On average, 
8-10 people were scheduled per session, with attrition and no-shows bringing actual 
attendance per session to about 5-6 people. At some conferences additional recruitment 
was done on site, via fliers and word-of-mouth. 

The major restriction on the number of conferences attended was one of timing. The 
organizations national conference had to occur after the Definition Panel meeting in 
January 2005 but before mid-summer, in order to have the results compiled for the 
Definition Panel’s second meeting in September 2005. 

Targeted educational conferences were the following (with participant counts indicated): 

Gouncil for Exceptional Ghildren (GEG) — 6 sessions, 35 people 

American Educational Research Association/National Gouncil on Measurement in 

Education (AERA/NGME) — 3 sessions, 17 people 

International Reading Association (IRA) — 5 sessions, 24 people 

Gouncil of Ghief State School Officers (GGSSO) — 4 sessions, 20 people 

Society for the Scientific Study of Reading (SSSR) — 5 sessions, 19 people 

Learning Disabilities Association of America (LDAA) was initially scheduled for 
its conference held in early Eebruary, but due to the definition still being in flux at 
that time focus groups were unable to be held. As the time approached for the next 
two conferences (GEG and AERA/NGME), it was decided to take all three working 
definitions to the focus groups, instead of a single definition with proficiencies as had 
been originally planned. This allowed greater opportunity to gain feedback on several 
approaches to defining an accessible reading construct. 

With many key organizations not represented by the face-to-face groups, the decision 
was made to hold teleconference focus groups with many of the specific disability groups 
that are represented on the General Advisory Gommittee. Participants were identified 
by GAG members, and generally a single group was scheduled. These sessions were 
held in June and July. Between five and eight people were invited to participate for each 
teleconference focus group session. Due to scheduling difficulties, these sessions often 
had small samples. 
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Targeted disability organizations were the following (with participant counts indicated): 

National Down Syndrome Society (NDSS) — 4 people 

Center for Applied Linguistics (CAL) — 1 person 

Parent Advocacy Center for Educational Rights (PACER) — 3 people 

The American Speech-Language-Hearing Association (ASHA) — 2 people 

Gallaudet Research Institute — 4 people 

The Association of State Consultants of Blind/Visually Impaired — 6 people 
TASH/ARC — 7 people 

Sessions were scheduled for one hour, with actual discussion time set for about 45 
minutes. 

The protocol included session ground rules, project background information, and a set 
time schedule to provide uniformity from group to group. All but two of the face-to-face 
sessions were facilitated by the same person, and all of the teleconference sessions were 
facilitated by one person (who also had facilitated the remaining face-to-face sessions). 
All sessions were taped on audio cassette. In addition, a note taker was present for each 
session to document the proceedings. 

All participants were required to fill out a consent form before the session started, as 
well as a brief questionnaire that was given at the end of each session (via Internet for 
teleconference participants). Once the completed questionnaire was handed in, face-to- 
face participants were given a $50 gift card and a NARAP brochure, and teleconference 
participants were given a $25 gift certificate that could be redeemed via the Internet. 

All three definitions were presented to the groups for discussion after the ground rules 
and project background were reviewed. Although the goal was to discuss each definition 
on its own merits before comparing them, in many sessions the comparison began as 
early as during the discussion on Definition B. Eor the majority of sessions the discussion 
moved ahead with little prompting from the leader, and most attendees participated fully 
in the discussion. In addition, the participants often had questions about the project after 
the session officially ended and expressed interest in the final results. 



Results 

The results presented here focus on general issues rather than on specifics of wording. 
Comments from participants covered all types of topics, from very specific suggestions 
to broader ideas and more conceptual concerns. Our interest here was on those 
comments that addressed the definitions overall, and any specific themes in comments 
that related to students with disabilities. 

All three definitions that focus group participants discussed were in the same general 
structure. Each started with an overarching statement of what reading is, followed by 



4 



Focus Group Reactions to Three Definitions of Reading 



a description of how the process of reading occurs — including specific references to 
disability concerns — and concluded with a statement that clarified how understanding 
is obtained and affected. Using that structure as a guide, some common themes can be 
identified. 

Although we talked to many people with varied backgrounds, in general they formed 
four major groups: teachers/ specialists who work with students with disabilities, reading 
teachers/ specialists, reading researchers, and measurement/assessment experts. In 
general the reading researchers’ interpretation of the language used in the definitions 
seemed to more closely align with the Definition Panel’s intended meaning. The test 
development experts viewed the definitions more through the lens of an eventual test 
and its impact on curriculum and teaching, being more concerned with terms and 
balance than the other groups. Teachers also were concerned about the influence on 
curriculum, but focused more on the impact the definition would have on the students 
they were familiar with. Those concerned with the reading outcomes of students with 
learning disabilities and mental retardation stressed the importance of finding ways 
to measure their student’s abilities and growth over time, even if the students could 
not meet many of the standard expectations for reading and comprehension. Those 
concerned with sensory disabilities such as deafness and blindness focused on where 
the definition may disadvantage their students, or where they thought the language 
used was problematic. There was a strong desire among the teachers of students with 
disabilities that affect reading to be able to demonstrate that their students have the 
ability to read and comprehend text, if presented in a way that was accessible. Those who 
worked mostly with non-disabled students seemed most interested in the complexities of 
understanding and the goals of more advanced reading. 

Overarching statement 

Most people preferred having the main emphasis in the definitions be placed on 
understanding and did not think that it was appropriate to have decoding appear equal 
to understanding in importance, as was implied when both were listed in the overarching 
statement. The teachers and measurement experts generally thought that understanding 
was the ultimate goal, and decoding was secondary to it, and consequently responded 
more favorably when decoding was not in the lead sentence. Reading researchers were 
more likely to view decoding as equally important and many appreciated its inclusion, 
although even this population viewed understanding as the more important outcome. 

Definition of decoding 

The term decoding was viewed differently by the reading researchers than the other 
groups. The researchers viewed decoding as a more comprehensive term for the 
process of reading that included many aspects of how people turned text into language. 
Conversely many teachers viewed decoding as too simple a term to represent the 
technical process of reading fully — often limiting the term to the process of “sounding 
out” a word and relating the term to phonics in many instances. Those who defined 
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decoding that way often would conclude that decoding represented too low-level a 
skill for many readers. Much of the dislike for the inclusion of decoding as equal in 
importance to understanding seemed to stem from this difference in the scope of what 
decoding represented. 

Definition of understanding 

There was often discussion on the relative nature of the terms understanding and 
meaning (which were used in the definitions) and comprehension (which was not used). 
Many felt the three terms were not completely interchangeable and implied different 
issues. Although the nuances of the three terms were not always consistent from person 
to person, it most commonly seemed to he a concern that the term used should not limit 
the scope to simple ideas and concepts found in text hut more complex components 
as well that went beyond facts. If put in a hierarchy, “understanding” was viewed as 
the most simplistic (e.g.,. word definitions), with “comprehension” the next level (e.g., 
integrating all the words into a coherent whole), and “meaning” the highest (e.g., 
including subtle nuances and deeper context). 

Two different descriptions of how understanding is impacted for students were included 
in the definitions. One listed components that affect understanding and was generally 
viewed as positive, although many suggestions were made for additional concepts 
— most commonly background knowledge and culture — to be included. The other 
was less specific, and while some preferred the use of the term constructive processes 
as being inclusive enough, others felt that was too vague, particularly the measurement 
and general researchers. Differences between deriving and constructing meaning were 
suggested as well, with the notion that deriving meaning may imply a specific intent to 
the text that the reader needs to identify while constructing meaning allows for more 
outside influences that the reader brings to the text. 

Use of the terms "speech/spoken words" 

Almost all groups objected to the references to speech and spoken words. These were 
viewed as problematic to students who had no spoken language — most commonly 
identified as students who are deaf or hearing impaired, although students with autism 
were also mentioned at times as readers who may be non-verbal. 

In addition, the teachers often interpreted “translating text to speech” as being specific 
to oral reading (reading out loud). And much like the objections to decoding, the issue 
seemed to be that such behavior (sounding out, reading out loud) was not a process for 
advanced readers. However, the reading researchers (and some of the others) were more 
likely to view the terms as internal speech, or representative of the process of turning text 
into language, for which speech is a proxy. 
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Braille 



The inclusion of braille was supported as simply being the version of text accessible 
to those students who read braille. Classifying it as an adaptation or accommodation 
was questioned by some. The use of a read aloud accommodation instead of braille 
was mentioned a few times for students who either had not, could not, or would not 
learn braille. Whether a read aloud was an acceptable means to measure reading 
comprehension for blind and visually impaired students was a topic where there 
appeared to be little consensus, although those in our focus groups who worked with 
blind students clearly supported braille as the preferred mode. 

Auditorization 

Auditorization got a mixed response. Many believed that including auditorization in the 
definition undermined a basic construct of reading which includes the interpretation 
of text. Although some, mostly the teachers with students with disabilities, argued that 
if understanding was the main goal then it could be appropriate as a means to measure 
understanding apart from reading processes. Intriguingly some of the participants who 
argued that comprehension was the main goal and were opposed to decoding being 
included, reconsidered when presented with the possibility of measuring reading with 
an audio assist. The discussion on this topic naturally led to where reading ends and 
listening begins. In general, support for auditorization was limited, with the likelihood of 
approving of auditorization being negatively related to one’s approval of decoding in the 
definition. 

Disconnect between understanding and decoding 

The teachers of students with disabilities were often concerned that many of their 
students, particularly those with learning disabilities, could show skill in decoding 
but have no understanding of what they read. Conversely, they were concerned about 
students who were capable of understanding but could not read well. The similarity of 
these students with ESL students was mentioned often. 

Other issues 

The nature and scope of the term text was often discussed at length. For many, text is a 
highly inclusive term that can include pictures, graphs, tables, numbers and so on, while 
for others it is restricted to letter symbols. Similarly, when reading ends and literacy 
begins was also a topic of conversation. Many thought a definition only of reading was 
not inclusive enough of the overarching goal of communication. The participants who 
focused more on a tighter focus on text and reading seemed happier with the definitions, 
while those with a broader focus on representations of meaning and literacy were not as 
pleased with the definitions. 
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In addition to the focus on the construct of reading as set forth in the definitions, there 
was also a focus on specific issues relevant for the various disability groups. Themes that 
emerged from disability-specific focus groups are found below, and they corresponded 
highly with themes that emerged from conference participants. In addition to general 
themes, however, focus group participants mentioned a plethora of aspects of reading 
related to students with specific types of disabilities. Although the samples of participants 
were small, the results provide a strong addition to the disability information obtained 
from the larger in-person focus groups. 

Down Syndrome 

Although readers with Down Syndrome do learn to read by decoding, some readers with 
Down Syndrome are non-verbal; for them, the process of translating “text to speech” 
is not relevant. Because reading is a visual endeavor for most students with Down 
Syndrome, “auditorization” should be considered an adaptation, not part of the reading 
process itself 

Emotional/Behavior Disorders 

Comprehension is the biggest issue for the population of students with emotional or 
behavioral disabilities. Many students decode text just fine, but do not understand the 
meaning of text. Other factors, such as memory, fluency, and vocabulary may affect the 
comprehension levels of this population, and should be included in any definition. 

Mental Retardation 

Some readers with mental retardation have difficulty decoding, but can understand text 
through other strategies. Readers with mental retardation need to be engaged in order 
to succeed. Struggling readers may quickly give up if text is not interesting or relevant to 
their lives. Readers with mental retardation may be non-verbal; therefore, an expectation 
of translating text to speech is unreasonable. Comprehending text (by a variety of means) 
is the most important goal of reading for students with mental retardation. 

Learning Disabilities 

Some readers with learning disabilities use alternative approaches to reading, such 
as screen readers or books on tape, but still consider the process “reading.” A focus 
on accessing information, rather than individual skills, is most appropriate for this 
population. Accommodations, such as “auditorization” are commonplace in higher 
education, but rarely found in K-12 education. 

Speech/Language Impairments 

Readers with speech/language impairments may not translate text to speech as part of 
the reading process. Measures of fluency (for either silent reading or reading aloud) must 
include a focus on both fluency and morphological processing in order to truly measure 
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the reading abilities of students with speech language impairments. 

Blind 

According to participants, braille is an equivalent system of writing to print. Participants 
also noted that 40% of people who are blind have some other form of disability. Some 
may not have speech, so translating text to speech may be impossible. Text in auditory 
formats is used by blind populations, but should be used with extreme caution, because it 
may lead to a decrease in the teaching of braille. All definitions should include decoding 
but should be strongly centered in the derivation of meaning from text. 

Deaf 

Students who are deaf typically do not “decode” because they often do not have 
phonological skills. This population also does not translate text to speech. Definitions for 
students who are deaf should be more comprehension-based and less skills-based. It was 
proposed by some participants that students who are deaf should be viewed as similar 
to ESL students as their communicative language is not English and American Sign 
Language does not map in terms of grammar and vocabulary to the English language. 



Conclusion 

Participants in focus groups strongly favored definitions of reading that centered on 
understanding of text. Participants were wary to endorse definitions that appeared 
overly-focused on decoding, although decoding was considered an important aspect 
of reading by most participants. A majority of participants disliked overt references to 
translating text to speech, either as an important function of reading or as impossible 
for some students with disabilities. Many participants were hesitant (or completely 
disagreed) that auditorization could be considered reading. Pinally, participants 
offered a variety of explanations about how students with disabilities read. Although 
approaches varied from disability to disability, overall themes reflected the main finding 
that understanding is the foremost aspect of reading upon which researchers and test 
designers should focus. 
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